Global-Local Enhancement Network for NMF-Aware Sign Language Recognition

نویسندگان

چکیده

Sign language recognition (SLR) is a challenging problem, involving complex manual features (i.e., hand gestures) and fine-grained non-manual (NMFs) facial expression, mouth shapes, etc .). Although are dominant, also play an important role in the expression of sign word. Specifically, many words convey different meanings due to features, even though they share same gestures. This ambiguity introduces great challenges words. To tackle above issue, we propose simple yet effective architecture called Global-Local Enhancement Network (GLE-Net), including two mutually promoted streams toward crucial aspects SLR. Of streams, one captures global contextual relationship, while other stream discriminative cues. Moreover, lack datasets explicitly focusing on this kind feature, introduce first non-manual-feature-aware isolated Chinese dataset (NMFs-CSL) with total vocabulary size 1,067 daily life. Extensive experiments NMFs-CSL SLR500 demonstrate effectiveness our method.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Sign language perception research for improving automatic sign language recognition

Current automatic sign language recognition (ASLR) seldom uses perceptual knowledge about the recognition of sign language. Using such knowledge can improve ASLR because it can give an indication which elements or phases of a sign are important for its meaning. Also, the current generation of data-driven ASLR methods has shortcomings which may not be solvable without the use of knowledge on hum...

متن کامل

MAN-MACHINE INTERACTION SYSTEM FOR SUBJECT INDEPENDENT SIGN LANGUAGE RECOGNITION USING FUZZY HIDDEN MARKOV MODEL

Sign language recognition has spawned more and more interest in human–computer interaction society. The major challenge that SLR recognition faces now is developing methods that will scale well with increasing vocabulary size with a limited set of training data for the signer independent application. The automatic SLR based on hidden Markov models (HMMs) is very sensitive to gesture's shape inf...

متن کامل

Investigating NMF speech enhancement for neural network based acoustic models

In the light of the improvements that were made in the last years with neural network-based acoustic models, it is an interesting question whether these models are also suited for noise-robust recognition. This has not yet been fully explored, although first experiments confirm this question. Furthermore, preprocessing techniques that improve the robustness should be re-evaluated with these new...

متن کامل

Sign Language Recognition

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gest...

متن کامل

Visual Sign Language Recognition

We have developed the Hand Motion Understanding (HMU) system that understands static and dynamic signs of the Australian Sign Language (Auslan). The HMU system uses a visual 3D hand tracker for motion sensing, and an adaptive fuzzy expert system for classification of the signs. This paper presents the hand tracker that extracts 3D hand configuration data with 21 degrees-of-freedom (DOFs) from a...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: ACM Transactions on Multimedia Computing, Communications, and Applications

سال: 2021

ISSN: ['1551-6857', '1551-6865']

DOI: https://doi.org/10.1145/3436754